Artificial Intelligence and Wittgenstein's use theory of meaning

AI’s view:

what is AI

Artificial intelligence means using program to accomplish tasks human can do by simulating human’s brain. The purpose of artificial intelligence is to bypass the brain and body to achieve a full understanding of human intelligence. The idea can be traced back to Turing machine (we can already see some interesting coincidence here~ Turing has attended Wittgenstein’s class in Cambridge, though Turing holds different opinions, Wittgenstein values his ideas very much. Maybe Turing also get some inspiration during the idea conflict.)

In 1956, H. Simon and others designed the first AI program called “logical theoretician”. It can prove 38 of the first 52 theorems in “Principles of Mathematics.”

Today, AI has thrived in research, application, and education field in computer science and other disciplines. It specializes in issues such as procedural languages, methods of reasoning and problem solving, visual identification, and expert systems. The most powerful use of nowadays AI is in computer vision.

AI tries to help humans understand the human intelligence by means of symbolic operations, so it proposes under what conditions we have reason to attribute the state of mind to a purely physical system. It has made a great contribution to the development of cognitive science and the philosophy of the mind. There is a distinction between “strong AI theory” and “weak AI theory.” The weak AI theory aargues that computer programs help to understand the state of mind. This point has been widely accepted. The strong point of view claimed that the “heart” of the computer exemplifies the human psychological process. This argument is controversial. Searle argues that, unlike people, the syntactic manipulation of symbols by machines is not accompanied by a semantic understanding of the meaning of symbols.

Why Wittgenstein’s theory should be related to AI

The AI discipline was formally established until the Dartmouth Conference in 1956, at that time Wittgenstein had already passed away. However, this does not mean that, from his discussion of other philosophical issues, we cannot find its philosophical inspiration for artificial intelligence, and on this basis reconstruct an artificial intelligence using Wittgenstein’s philosophy.

Artificial intelligence does not have a certain level of unity as mature disciplines such as physics and chemistry. In other words, the fundamental issues related to the basic definition of disciplines, such as what is “intelligence” “what is artificial intelligence,” and “what is the basic means of artificial intelligence”, the divergence among different AI schools is almost impossible to reconcile (until today It is far from being able to reach a consensus on how artificial intelligence can be realized. No one has a effective plan to achieve it.

One of the hopes of the public in philosophical research is that philosophy should be able to provide some kind of “integrated” solution to conflicts between discourse systems of various natural sciences (even between science and common sense). For example, philosophy of physics should provide some reconciliation solutions for conflicts between quantum mechanics and relativity; and cognitive science philosophy must provide some reconciliation solutions for conflicts between common-sense psychology and neuroscience; as for metaphysics It is necessary to provide a reconciliation program on the topic of “what is there?” and the conflict between common sense insights and scientific perspectives in the most general sense. From this point of view, facing the chaotic situation of within artificial intelligence discipline, people naturally expect philosophers to do something for them.

I think Wittgenstein’s philosophy may has closest relation to artificial intelligence and it can also give people inspiration on AI.

The pursuit of Unity in Wittgenstein’s Philosophy

Wittgenstein’s philosophy seems to have an inherent “integrated” quality, that is, its philosophy is always trying to provide a unified solution to all problems in the world. This academic style is scarce for the present situation of scattered research on AI. (actually, due to the rising of probability and statistics methods in AI, we focus less on the ideas and essence of AI, but only on techniques of math.)

This philosophical temperament was embodied in Wittgenstein’s Early work(TLP). In the “preface” of TLP, he wrote a bit arrogantly:

“……On the other hand the truth of the thoughts that are here communicated seems to me unassailable and definitive. I therefore believe myself to have found, on all essential points, the final solution of the problems. And if I am not mistaken in this belief, then the second thing in which the value of this work consists is that it shows how little is achieved when these problems are solved”

The broad coverage of this book undoubtedly confirms his rhetoric. Within just over 20,000 words, Wittgenstein actually involved three topics: the metaphysical construction of the world and the metaphysical world. Linguistic representations, and the “right of being silence” for matters that cannot be characterized. These three topics basically correspond to the three steps of the “knowledge representation” task of artificial intelligence science:

• First, the metaphysical understanding of the object being represented;
• Second, the technical means (especially the logical technical means) of knowledge representation
• Third, the problem of delineation of the possibility boundary of the scope of knowledge representation (under the premise of selecting a specific representation method).

From the perspective of the history of AI development, Wittgenstein ’ thought roughly corresponds to the “primary physics plan” of Hayes, who once attempted to use predicate calculus as a technical means to compile human daily physics knowledge (in contrast to scientific physics) into axioms and set certain inference rules so that the system can automatically infer the required conclusions in the relevant context.

In other words, early Wittgenstein, like Hayes, tried to use the “once-for-all” approach to complete the task of comprehensive representation of daily knowledge on the premise of formal logic. They also assumed that in this process of knowledge representation, any particular agent that uses this knowledge is either negligible (as in Hayes) or cannot be characterized (in Wittgenstein, the “subject” merely “shows” rather than “characterizes” the boundaries of the linguistic representation). Although Wittgenstein eventually gave up the TLP system, and Hayes’s “simple physics” plan has now become part of AI history, they share the same kind of theoretical courage to try to integrate all aspects of daily knowledge. And ambition, is what today’s AI engineers lack. Today’s AI theoreticians and engineers often abandon the interpretability of the model but pursue the increase in the accuracy of the model. This is an unavoidable but pitiable solution to solve complex problem in daily life. Nowadays AI already loses its interpretability under simple tasks. Could it become interpretable under hard tasks? Or maybe we can only solve the hard problem and produce real AI when we integrate everything again.

This kind of advocating comprehensive temperament has not disappeared in Wittgenstein’s later work Philosophical Investigations. Judging by appearance, this book is only constituted by a few scattered reviews of “philosophical sentiments”, but the “sloppyness” does not obscure its substantial comprehensiveness. In the preface of the book, he recognizes:

“The thoughts that I publish in what follows are the precipitate of philosophical investigations which have occupied me for the last sixteen years. They concern many subjects: the concepts of meaning, of understanding, of a proposition and sentence, of logic, the foundations of mathematics, states of consciousness, and other things.……Originally it was my intention to bring all this together in a book whose form I thought of differently at different times.……After several unsuccessful attempts to weld my results together into such a whole, I realized that I should never succeed.”

Wittgenstein uses language games to settle everything

The topics involved in the “Philosophical Investigations” are essentially related to each other, although the author has to observe the same or almost the same entity from different perspectives, and draw new sketches.

If we summarize the subject of “Philosophical Investigations” in the most succinct way with AI colors, then we might say that this book discusses agents: Under what kind of normative constraints, in a dynamic diachronic environment, using relevant characterization tools (especially daily language) to perform causal interactions with the environment and other agents, and ultimately accomplish certain tasks.

Obviously, because the interactive process between the agent and the environment (using language) is very complicated, Wittgenstein had to carefully philosophize the normative conditions of the various links involved in the process. For example, his reflection on the concept of “understanding” and the concept of “consciousness” is about the agent itself; the investigation of concepts such as “logic”, “proposition”, and “meaning” is about representation of mediation; and discussion about “life “Formal” involves the portrayal of an external environment grasped by an agent.

Wittgenstein put all these discussions together in the general framework of language games and settled them one by one. By constantly exploring new varieties of language games, he eventually turned his entire post-philosophy into an unusually open system: this system has no clear boundaries and it can extend its jurisdiction to new virgin lands at any time.

It is really hard to put all these things together, but it seems that it is the best way to achieve real artificial intelligence. Nowadays many “naive” researchers are immersed in tuning there networks to do better at image classification、object recognition or related “simple/single task”. But it is far from the real intelligence. Recently some excellent scientists concer more about how to produce a real agent, but it is hard to use current state-of-art deep learning architecture to design a real agent. So deep learning cannot play hard games( game like GO looks hard to human, but it is more of a computational work to computer, so it can’t reflect the real intelligence level of computer).

Real-time information processing when using it

From the AI’s point of view, Philosophical Investigations surpass TLP because it no longer regards the static knowledge system as a focal point of philosophical theories, but shifts the focus to the actions of agents. Moved to the real-time processing of information. In other words, the issue of subject portrayal in the marginal position of its early philosophy has now become one of the focuses of the study. Considering the diversity of different real-time information processing tasks that different agents face, the complexity involved in Philosophical Investigations naturally will be far better than the TLP - which also partly explains why the late Wittgenstein’s philosophy is so complicated. In AI textbooks, the problem solving task also correspond to completely different problem solving techniques! (For example, you can use Bayesian networks for reasoning, use genetic algorithms to do optimization of solution selection, use neuron networks for pattern recognition, use symbolic AI techniques for expert systems, etc.). Wittgenstein used a “language game” cover to try to put some of its most elementary neutrality on the various information processing processes. In the fragmented AI kingdom, we have seen this kind integration.

Wittgenstein’s “Factualism” and use theory

From this point of view, Wittgenstein is indeed a different kind of alternative (or, he is an anti-analytic philosopher in analytic philosophy, and an analytical philosopher in anti-analytic philosophers). This special philosophical character mainly reflects in the atmosphere of some kind of “Factualism” by his writings.

“Factualism is referring to a philosophical style in which philosophers are to faithfully describe the known areas of humanity. In other words, they should not be as ignorant of the linguistic phenomena of everyday language as the traditional rationalist philosophers. In other words, “Factualism” must balance the “respection of people’s existing experience” and “rethinking of the existing human experience”, and thus gradually tease out the deep formal framework from the complex human experience.

From the author’s point of view, the following paragraphs written by Vickers in Philosophical Studies fully demonstrate this practical essence of Vickers’ later philosophy—and it is in a way that makes AI experts happy:

Think of the tools in a toolbox: there is a hammer, pliers, a saw, a screwdriver, a rule, a glue-pot, glue, nails and screws. a The functions of words are as diverse as the functions of these objects. (And in both cases there are similarities.)

Of course, what confuses us is the uniform appearance of words when we hear them in speech, or see them written or in print. For their use is not that obvious. Especially when we are doing philosophy!

This corresponds to the programming design level of programming

In the language of AI, Wittgenstein actually distinguishes between the two interfaces of language operation:

• the usage of the terms between the words “user-friendly” is not obvious, and the traditional philosophers are also confused by its similar surface.
• In a deeper “programming level”, the differences between the functional structures of the words are presented - in which each functional structure corresponds to a different input-output relationship and different input-oriented operations.

Obviously, in Wittgenstein’s opinion, the real task of philosophy is to find the differences between these functional uses as marked by words—or to find real programs that allow our natural language mechanisms to flourish. This is the true ground that philosophers want to hug, or the “pragmatic facts” faced by the “Factualer”.

This position of Wittgenstein’s inspiration for today’s AI is: If an AI engineer wants to faithfully reproduce intelligent activities on the symbolic representation level, his working sample can neither be too close to the surface form of human natural language, nor too far away.

Not too close is because the surface form of natural language does not show the real mechanism of the use of words; not too far away is because the somewhat confusing “user-friendly” interface, after all is the starting point to explore the real mechanism behind it.

The recommended working procedure should be: The researcher must first observe the different functional roles played by the similar “user interface” in different use environments (ie, the input-output relationship), and then re-introduce the most likely programs. Then find the most appropriate engineering simulation method in the existing “arsenal library” of computer science.

This means that the engineering reproduction of the language operation mechanism is essentially a “top-down” construction process that use the experience of human natural language mechanisms.

At the initial stage of this construction process, that is, at the stage of qualitatively characterizing the operating mechanism of the characterization system, the reference significance of Wittgenstein’s philosophy will be most clearly demonstrated.

Additional: Basic data mining of Philosophical Investigations

I also spent some time analyzing the context of Philosophical Investigations using python, since it is more casual and less organized, it seems it’s hard to learn many information from it.

prepare data

import numpy as np
import jieba
import os, codecs  
from collections import Counter  
import matplotlib.pylab as plt
plt.style.use('ggplot')
import pandas as pd
import matplotlib
import re
with codecs.open('text.txt', 'r', 'utf8') as f:  
    txt = f.read()
def get_words(txt):  
    seg_list = jieba.cut(txt)  
    c = Counter()  
    for x in seg_list:  
        if len(x)>1 and x != '\r\n':  
            c[x] += 1  
    return c  # c.most_common(100)
if __name__ == '__main__':  
    with codecs.open('text.txt', 'r', 'utf8') as f:  
        txt = f.read()  
    a = get_words(txt)
dat = a.most_common(1000)
countlist = np.ndarray([1000,2]).astype('str')
for i in range(1000):
    countlist[i][0] = dat[i][0]
    countlist[i][1] = dat[i][1]
wordlist = np.array(re.split(' |; |, |\*|\n',txt))
select_ind = np.where(np.isin(countlist[:,0],np.array(selectwords)))[0]
countlist_ = []
for i in range(1000):
    countlist_.append(countlist[i,0]+': '+countlist[i,1])
pd.DataFrame(np.array(countlist_)[:200].reshape(20,10))

words counts, most frequent

count = 27
fig,ax=plt.subplots(1,figsize=(20,10))
ax.bar(range(count),countlist[:count,1].astype('int'))
ax.set_xticks(range(count))
ax.set_xticklabels(countlist[:count,0])
ax.set_title('most frequent words')

most frequent words with specific meaning

count = 50
index = index_54[:count]
fig,ax=plt.subplots(1,figsize=(20,10))
barlist = plt.bar(range(count),countlist[index,1].astype('int'))
for i in np.arange(0,2):
    barlist[i].set_color('r')
for i in np.arange(2,5):
    barlist[i].set_color('b')
for i in np.arange(5,12):
    barlist[i].set_color('g')
for i in np.arange(12,26):
    barlist[i].set_color('m')
for i in np.arange(26,count):
    barlist[i].set_color('y')
ax.set_xticks(range(count))
ax.set_xticklabels(countlist[index,0],fontsize=15)
ax.set_title('Most Frequent Words in Philosophical Investigations',fontsize=30)
plt.setp(ax.get_xticklabels(), rotation=30, horizontalalignment='right')
fig.tight_layout()

words frequency by chapter

chapterind = [1491,82898]
namelist = countlist[index,0]
def count_frequent(chap,count):
    freqlist =[]
    if chap <1:
        for i in range(count):
            freqlist.append(np.where(wordlist[chapterind[chap]:chapterind[chap+1]] ==namelist[i])[0].shape[0])
    else:
        for i in range(count):
            freqlist.append(np.where(wordlist[chapterind[chap]:] ==namelist[i])[0].shape[0])
    return np.array(freqlist)
freq_var = np.ndarray([2,50])
for i in range(2):
    freq_var[i] = count_frequent(i,50)
freq_var[0,:] =freq_var[0,:]/len1
freq_var[1,:] =freq_var[1,:]/len2
fig,ax=plt.subplots(1,figsize =(10,10))
ax.matshow(np.repeat(freq_var.T,4).reshape(50,-1) ,cmap ='jet')
ax.set_title('50 Key Words Fluctuation in 2 Parts')
ax.set_xticks(np.arange(0,12,6))
ax.set_xticklabels(['Part I','Part II'])
ax.set_yticks(range(50))
ax.set_yticklabels(namelist)

You can see the words variation in different chapters. Unfortunately, unlike TLP, Wittgenstein only divide his book into two parts, so we can learn very little by feature fluctuation.

Calculate different words’ distances

I also think about how to depict different words’ relationship. One easiest way I think of is calculating two words minimum distance in the book whenever they appears. For example:

def calculate_distance(ind1,ind2):
    pos1 = np.where(wordlist==namelist[ind1])[0]
    pos2 = np.where(wordlist==namelist[ind2])[0]
    num1 ,num2 = pos1.shape[0],pos2.shape[0]
    if num1>num2:
        small = num2
        large = num1
        lararr = pos1
        smarr = pos2
    else:
        small = num1
        large = num2
        lararr = pos2
        smarr = pos1
    disarr = np.ndarray([small,large])  #each line calculate the small set's ith word's and large set's every words distance
    arr1= np.repeat(smarr,large).reshape(-1,large)
    arr2= np.repeat(lararr,small).reshape(-1,small).T
    mindis = np.min(np.abs(arr2-arr1),axis=1)
    return mindis
def draw_dist_count(ind1,ind2):
    fig,ax=plt.subplots(1,figsize=(20,10))
    ax.bar(range(calculate_distance(0,1).shape[0]),calculate_distance(0,1),color='g')
    ax.set_title('Minimum Distance of '+namelist[ind1]+" and "+namelist[ind2])
draw_dist_count(0,1)

It is the minimum distance of two words: picture and game, we can see at the first half of the book, the two words have closer distance, which means they have higher chance appearing together or nearby. But in the last half, the author didn’t arrange them appearing together very close.

We can also see one word’s distance with many other words: game and other words

fig,ax=plt.subplots(4,2,figsize=(20,20))
for i in range(4):
    for j in range(2):
        ax[i,j].bar(range(calculate_distance(2*i+j,8).shape[0]),calculate_distance(2*i+j,8))
        ax[i,j].set_title('Minimum Distance of '+namelist[9]+" and "+namelist[2*i+j])

As we can see, the word game’s appearance with other words have different patterns, for example, it is closer to word use than word picture.

We can also overview one word’s distance(relationship) with others using hist or boxplot

fig,ax=plt.subplots(4,2,figsize=(20,20))
for i in range(4):
    for j in range(2):
        ax[i,j].hist(calculate_distance(0,1+2*i+j),bins =50,color='b',alpha=0.4)
        ax[i,j].set_title('Minimum Distance of '+namelist[0]+" and "+namelist[1+2*i+j])

dist_data = {}
for i in np.arange(1,20):
    dist_data[i] = calculate_distance(0,i)
dataframe_dxp = pd.concat((pd.DataFrame({namelist[i]:dist_data[i]}) for i in np.arange(1,20)),axis=1)
import seaborn as sns
fig, ax = plt.subplots(figsize=(100,20))
sns.boxplot(data =dataframe_dxp,ax=ax,boxprops=dict(alpha=.5),color='g')
ax.set_title(u'Proposition and others',fontsize=120)
ax.set_xticks(range(19))
ax.set_xticklabels(namelist[1:20],fontsize=80)
plt.setp(ax.get_xticklabels(), rotation=30, horizontalalignment='right')
fig.tight_layout()

That’s all about my basic data mining and analysis of Philosophical Investigations. It took me a some time writing codes to count and plot. I believe in the future we can use data mining, natural language processing to do analysis more automatically. What’s more, it may do some really exciting and serious study instead of my basic plotting. We may use the powerful so called “artificial intelligence” tool to mine thousands of books and materials to analyze the hidden ideas which is omitted by human due to our limitation of memorizing. Thus we can understand more about the author and the ideas behind the book. For some book as “random and complex” as this one, it may be a good tool to help us understand more by using statistics and visualization.